UNCLASSIFIED 


AD  NUMBER 


AD282767 


NEW  LIMITATION  CHANGE 
TO 

Approved  for  public  release^  distribution 
unlimited 


EROM 

Distribution  authorized  to  U.S.  Gov't, 
agencies  and  their  contractors; 
Administrative/Operational  Use;  AUG  1962. 
Other  requests  shall  be  referred  to  U.S. 
Army  Biological  Laboratories^  Frederick^ 
MD. 


AUTHORITY 


BORL  D/A  Itr,  27  Sep  1971 


THIS  PAGE  IS  UNCLASSIEIED 


UNCLASSIFIED 


R0fMd44O0i 


K>n02:  Vhen  goveraaeat  or  dther  drsvlags^  speoi- 
fieatlons  or  other  date  ere  used  for  any  porpoae 
other  than  In  connection  vlth  a  definitely  related 
gorenaent  proeureoenc  operation,  the  U.  S. 
Qoveranent  thereby  Inciira  no  reaponelbiUty,  nor  any 
obligation  ehateoeverj  and  the  faot  that  the  Qovexn* 
Bent  aay  have  fozsBulated,  furnished,  or  in  any  eay 
supplied^  the  said  drawings,  speclfleationa,  or  'rther 
data  la  not  to  be  regarded  by  laplicatlon  or  other¬ 
wise  aa  in  any  Banner  lioensing  the  holder  or  any 
other  perscm  or  corporation,  or  conveying  any  rlj^ts 
or  pezBiBeion  to  BBsufaoture,  um  or  selX  any 
patented  invention  that  say  in  any  way  be  related 
thereto. 


TECHNICAL  AAANUSCRiPT  10 


A  METHOD  FOR 
SYSTEMATIC  KROR  ANALYSIS 
OF  DIGITAL  COMPUTER  PROGRAMS 


AUGUST  1962 


r;  ■  ' 

* 

t 


111 


o 


UNITED  STATES  ARMY 
BIOLOGICAL  LABORATORIES 
FORT  DETRICK 


U.S,  ABKY  CHSMXCAL-BIOLOOIOa-RADZOLOOICAL  A05KCV 
BIOLOGICAL  lABOKATCAZES 
Fort  Detrlek,  Maryland 


TIm  vork  raportad  faara  vai  parforaad 
undat  Projaet  4X99>26>001,  Taak  *021 
Baale  Bloaathaaatlea  kaaaareh.  H^a 
axpandltura  erdar  vaa  2038. 


Joan  C.  Millar 


Blonatheisatica  Division 
DIRECTOR  OP  TECHNICAL  SERVICES 


Project  4X99-26-001 


August  1962 


1 


Thla  docuoBBt  or  any  portion  tharaof  aay 
not  ba  reproduced  without  specific  authori¬ 
sation  from  the  CosEMnding  Officer,  Biolofi- 
cal  Laboratories,  Fort  Detrick,  Frederick, 
Maryland;  however,  A8TZA  is  authorised  to 
reproduce  the  doewsent  for  U.S,  flovetoasat 
pnr  poses. 


The  infOTBStion  in  this  report  has  not  been 
cleared  for  release  to  the  general  pub lie. 


ABTTA  AVAnA«Tt.rrY  WOTia 

(Qualified  requesters  aey  obtain  copies  of 
this  docuaant  from  A8TIA. 

Foreign  aanduneeaent  and  dlsssBinatlon  of 
this  docusent  by  AST lA  Is  United. 


Tha  author  vithaa  to  axpraai  lincare  appracUtion  to  Dr.  Clifford  J, 
Malonay;  Chiaf,  BlocaathaMtlea  Division^  for  hit  guidance  and  encouragement 
throughout  tha  oouraa  of  thla  project  and  for  hla  valuable  contrilutlons  to 
the  revlelon  of  the  aaauierlpt.  Gratitude  la  aleo  extended  to  Kr.  Truaan 
May,  whoae  eomputar  prograB  wee  uted  ae  a  baela  for  the  experimentation 
involved  in  developing  the  method  preeented  in  tbia  paper. 


A  method  ia  preeented  that  trill  afford  the  programmer  a  eyatematic  vay 
in  trhich  to  generate  e  teat  deck  eontatnlng  ell  poeelble  eombinatlRia  of 
input  to  a  given  problem.  After  eoaelderatlon  of  verioua  alternative  repre* 
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I.  INTRODUCTION 


Effective  program  check'<out  Is  imperative  to  any  complex  computer  pro¬ 
gram.  One  or  more  teat  cases  are  always  run  for  a  program  before  it  is  con¬ 
sidered  ready  for  application  to  the  actual  problem.  Each  test  case  checks 
that  portion  of  the  program  actually  used  in  its  computation.  Too  often, 
however,  errors  show  up  as  late  aa  several  months  after  a  program  has  been 
put  into  operation.  This  is  an  indication  that  portions  of  the  program 
called  upon  only  by  rarely  occurring  input  conditions  have  not  been  tested 
during  check-out. 

In  order  for  a  person  to  rely  with  confidence  upon  any  particular  pro¬ 
gram,  it  is  not  sufficient  for  him  to  know  that  the  program  works  most  of 
the  tinte  or  even  that,  so  far,  it  has  never  mad'  a  mistake.  The  real 
question  is  whether  it  can  be  counted  upon  to  fulfill  its  functional  speci¬ 
fications  successfully  every  single  time,  This  means  that  after  a  program 
has  paaeed  the  check-out  stage,  there  should  be  no  possibility  that  an 
unusual  culmination  of  input  data  or  conditions  may  bring  to  light  an 
unexpected  bug  in  the  program.  Every  portion  of  the  program  must  be 
utilised  during  check-out  in  order  that  its  accuracy  may  be  confirmed. 

The  purpose  of  the  work  reported  here  has  been  to  develop  a  systematic 
way  in  which  a  prograoBar  may  test  all  realistic  combinations  of  input 
data,  and  hence  all  portions  of  a  given  program.  Although  at  first  this 
seems  to  be  an  arduous  task,  its  handling  is  simplified  by  an  orderly 
approach,  and  its  reward  is  a  greater  assurance  of  the  accuracy  and  re¬ 
liability  of  the  program.  The  potentiel  number  of  test  cases  is  given 
by  2^,  where  B  is  the  nus^er  of  branch  points  in  the  flow  chart.  The 
computer  could  be  prograonad  to  generate  all  of  these  cases  automatically. 
However,  if  the  nusiber  of  branch  points  is  at  all  large,  the  total  nu&her 
of  potential  cases  would  become  impractical ly  large,  thou^  most  of  these 
would  be  unrealistic  or  identical  to  others  of  the  epses,  and  hence  un¬ 
necessary  to  test.  The  analysis  set  forth  in  this  paper  has  as  its  object 
the  determination  by  straightforward  cAans  of  the  full  list  of  poaaible— ' 
aa  opposed  to  potential— test  cases,  their  method  of  convenient  genera¬ 
tion,  and  their  use  in  the  location  of  specific  errors  in  the  program. 

The  approach  taken  here  of  using  varied  input  conditions  as  the  means 
to  insure  thorough  testing  of  the  program  is  particularly  applicable  to 
business  problems,  because  the  wide  variety  of  input  is  one.  of  the  major 
aspects  of  such  a  problem.  The  method  can  be  extended,  however,  to  deal 
effectively  with  scientific  problems,  even  though  some  of  their  branch 
points  will  not  be  dependent  upon  the  input  data.  The  procedure  should, 
therefore,  be  useful  as  a  method  for  systematic  error  analysts  of  computer 
programs  irrespective  of  the  type  of  program  to  be  analysed. 
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II.  BACKGROUND 


Probably  the  most  common  method  of  error  detection  is  the  running  of  a 
test  case  whose  answers  have  already  been  calculated  manually  for  com¬ 
parison,  Carefully  chosen  test  data  can  be  quite  successful  in  pointing 
up  the  major  errors.  Other  methods  of  error  prevention  that  can  be 
effective  are  (a)  to  code  into  the  program  certain  checking  devices^  such 
as  intermediate  check  sums,  and  (b)  to  write  the  code  in  such  a  way  that 
it  will  be  easy  to  check,  even  at  the  expense  of  certain  sophistications 
and  time-saving  devices. 1/*  It  is  always  hulpful  to  make  a  detailed  maniial 
check  of  the  code  as  written  prior  to  the  first  machine  run.  In  addition 
to  reviewing  the  coding,  a  second  flow  chart  can  be  drawn,  this  time  pre¬ 
pared  directly  from  the  coding,  to  determine  whether  the  program  executes 
its  instructions  according  to  Che  original  plan. 

An  increasing  number  of  check-out  methods  making  use  of  the  facilities 
of  the  machine  itself  are  being  employed.  Some  of  Che  more  common  methods 
are  manual  step-by-step  operation,  dumping  of  memory,  a  tracing  program 
with  automatic  skips,  and  Che  use  of  the  break-jump  switch,  which  will  scop 
the  program  and  Jump  to  any  specified  location.  Jacoby  and  Layton^'  have 
reported  on  an  automated  debugging  program  that  runs  the  program  to  be 
tested,  creating  a  trace  record  for  later  analysis  by  the  debugging  program 
itself.  Lois  Heibt,2y  has  described  e  very  intereating  program  that  enables 
the  computer  to  draw  multilevel  flow  charts  directly  from  the  coding  of  the 
problem.  A  particularly  promising  approach  to  checking  is  that  reported 
by  Senko.^  The  control  system  for  logical  block  diegnoiis  Chat  he  describes 
will  test  a  sub-routine  or  logical  block  of  instructions  as  a  separate 
entity,  executing  in  one  run  all  the  tests  neeeasary  to  explore  each  branch 
of  the  sub-routine.  The  significance  of  this  msthod  is  that'  it  allows  the 
programmer  to  supply  selected  test  date  at  different  points  throughout  the 
program  without  requiring  him  to  complicate  his  own  program  with  the  coding 
that  would  have  been  required  for  incorporating  loaders  into  the  original 
program. 


III.  NOTATIONAL  ALTERNATIVES 

As  an  aid  to  investigating  the  possibility  of  building  a  logically 
complete  test  deck  for  a  given  problem,  various  ways  were  considered  by 
which  the  logical  development  of  the  problem  might  be  displayed.  In  addi¬ 
tion  to  the  flow  chart  that  is  generally  employed,  a  number  of  alternate 
methods  of  representation  were  discovered,  including  matrix,  table,  out¬ 
line,  logical  tree,  and  symbolic  path  list.  Karpl/  has  discussed  the 
application  of  graph  theory  to  digital  computer  programming,  and  Voorhees^ 
has  taken  a  somewhat  similar  approach  in  considering  an  algebraic  formu¬ 
lation  of  flow  diagrams.  Since  the  notation  required  by  the  latter  methods 
differs  slightly  from  the  notation  chosen  here,  no  attempt  will  be  made  to 
Illustrate  their  similarity  to  the  other  representative  forms.  Including 


*  See  Literature  Cited. 
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the  flow  chart,  nine  methods  of  representation  will  be  described.  It  is 
possible  to  build  those  forms  either  directly  from  the  original  specifi¬ 
cation  of  the  problem  or  from  its  flow  chart.  For  the  purpose  of  illus¬ 
tration,  each  of  these  models  will  be  derived  from  an  illustrative  flow 
chart . 

In  the  flow  chart  of  Figure  1,  the  course  of  the  program  proceeds  in 
sequential  order  from  each  step  to  the  following  step  except  at  each  branch 
point,  where  one  of  two  available  paths  is  selected  according  to  the  re¬ 
sults  of  the  decision  at  that  point.  The  decision  elements  are  shown  as 
oval  boxes  and  are  designated  by  capital  letters;  the  alternative  results 
of  the  decisions  are  given  by  small  letters.  For  example,  if  "A"  sym¬ 
bolises  the  decision  element  [is  x  >  y?],  then  "a"  indicates  the  path 
taken  if  x  >  y,  and  "a"'  shows  the  path  followed  if  x  ^  y.  Numerical 
calculations  are  represented  by  rectangular  boxes. 

Since  the  logical  development  of  the  problem  is  affected  only  when  the 
program  is  required  to  select  between  two  possible  paths,  it  is  only  the 
decision  elements  that  need  concern  us  in  our  present  analysis.  For  this 
reason,  it  is  advantageous  to  consider  representations  of  the  problem  flow 
that  o^t  the  descriptions  of  the  numerical  steps. 

A  edified  flow  chart  from  which  the  numerical  calculations  have  been 
deleted  is  shown  in  Figure  2.  The  formulation  is  somewhat  sisdlar  to  the 
gra^  theory  form  used  by  Rarp.l/ 


Figure  2.  Modified  Flow  Chart,  Omitting  Numerical 
Calculations. 
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A  logical  tree  la  slfflilar  to  the  modlflud  flow  chatty  with  the  dis¬ 
tinction  that  a  tree  is  defined  as  "a  directed  graph  that  has  at  tsost  one 
branch  entering  each  node  and  contains  no  circuits. 1/"  Since  decision 
elements  E  and  G  can  each  be  entered  by  two  different  paths  in  Figure  2, 
the  results  of  those  decisions  each  appear  twice  in  the  tree  of  Figure  3. 
Although  the  tree  is  slightly  sore  cuobersome  to  draw,  it  has  the  advan¬ 
tage  that  each  path  is  clearly  visible,  and  the  number  of  paths  can  be 
determined  simply  by  counting  the  terminal  nodes  of  the  tree, 


©  ©  ©  ©  O  ©0  ®®  ® 

Figure  3.  Logical  Tree  Corresponding  to  Flow  Chart  of  Figure  1. 
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Outline  form  can  also  ba  used  to  depict  the  decisions  (Figure  4).  Each 
level  of  the  outline  is  considered  to  be  a  particular  decision  eletsent^  and 
the  two  entries  are  mutually  exclusive,  in  that  exactly  one  of  the  tvo  must 
be  chosen.  For  example,  decision  element  A  is  represented  by  the  Ror' ^ 
numerals  I  and  II.  If  the  program  does  not  select  I,  which  is  path  then 
its  only  other  choice  is  to  select  II,  which  is  path  "a'."  At  each  step  of 
the  outline,  the  appropriate  branch  choice  is  indicated  by  its  symbol  cor¬ 
responding  to  e  path  of  the  flow  chart  of  Figure  1. 


I. 

a 

A. 

b 

1. 

d 

2. 

d' 

S. 

b' 

1. 

e 

2. 

e* 

11. 

a' 

A. 

c 

1. 

a 

2. 

e' 

E. 

'  . 

c' 

1. 

f 

a. 

8 

b. 

8' 

2. 

, 

e. 

8 

b. 

8' 

Figure  4.  Calculation  Sequence  of 


Figure  1  Shown  in  Outline 
Form. 


In  the  table  of  Figure  5,  the  decision  elements  and  their  corresponding 
branch  choices  are  listed  sequentially  by  rows.  A  dash  in  any  position 
indicates  that  there  is  no  test  at  that  point  because  of  the  impossibility 
of  reaching  the  decision  element  for  the  row  by  the  path  selected  in  the 
preceding  rows.  Each  column  of  the  table  then  represents  one  particular 
path  of  program  flow.  This  tabular  representation  can  be  used  in  con¬ 
nection  with  the  Logic-Structure  tables  of  the  TABSOL  SyatemS^  of  program 
analysis.  Since  the  logical  development  of  the  program  is  completely 
designated  by  the  top  half  of  the  TABSOL  table  in  conjunction  with  the 
"go  to"  statements  at  the  end  of  the  columns,  the  third  and  fourth  quadrants 
of  TABSOL  may  be  omitted  when  the  tabular  form  is  used  for  error  analysis. 
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s  s  •  sea 

tea 

1  •  « 

a  a  f 

ass 

f 

f  f 

f 

G 

ess  ess 

t  •  • 

•  s  a 

. . . 

a  t  1 

S 

s'  g 

g' 

Figure 

5.  Tabular 

Form  of  Sample 

Computation, 

A  Bymbolic  path  list  (Figure  6)  is  simply  a  statement  d£  each  Indi¬ 
vidual  path  that  Is  a  part  of  Che  program.  It  can  be  vrltCet  directly 
from  the  flow  chart  or,  even  more  quickly,  from  the  tree,  outline,  or 
table. 


1)  abd 

2)  abd ' 

3)  ab'e 

4)  ab'e' 

5)  a'ce 

6)  a'ce' 

7)  a'c'fg 

8)  a'c'fg' 

9)  a'c'fg 

10)  a'c’fg' 

Figure  6.  Symbolic  Path  List  of 
Sample  Computet Ion. 


The  format  of  symbolic  logic  may  be  used  to  describe  these  paths  in  one 
statement,  but  the  result  seems  to  be  coo  compact  to  be  of  much  help  in 
error  analysis.  Writing  "v"  for  the  logical  "or"  and  the  absence  of  a  sign 
for  the  logical  "and,"  this  flow  chart  would  reduce  to 

a  [b(dvd  '  )vb  '  (eve  ' )  ]va  '  [c(eve  ')vc'(fgvfg'vfgvfg')]. 
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A  Boolean  matrix  (Figure  7),  A  ■  (*£<)>  can  be  very  helpful  in  de¬ 
scribing  the  flow  chart.  This  type  of  matrix  consists  entirely  of  O's  and 
I's.  In  the  connectivity  aatrix*^  for  the  flow  diagram,  aj_j  will  equal  1 
if  decision  element  j  is  the  next  consecutive  decision  element,  and  0 
otherwise.  In  this  matrix,  we  will  consider  the  decision  elemehcs  to  be 
represented,  in  alphabetical  order,  by  the  successive  rows  followed  finally 
by  Stop  2  and  then  Stop  3.  (For  the  reader's  convenience,  the  appropriate 
headings  for  rcwa  and  columns  have  bean  displayed  with  the  matrix.) 


stop 

ABCDEFG23 


Figure  7.  Boolean  Matrix  of  Sample  Calculation. 


A  limitation  of  use  of  the  Boolean  matrix,  when  considering  decision 
elements  only,  is  that  two  different  paths  from  one  decision  element  to 
the  next  one  cannot  be  illustrated.  Such  is  the  case  in  our  matrix  for 
the  two  paths  between  F  and  G.  In  the  matrix,  all  chat  is  shown  for  these 
two  paths  is  a  single  1  in  row  F  column  G.  A  refinement  of  the  Boolean 
matrix  can  be  accomplished  by  replacing  the  I's  by  the  notation  for  the 
actual  path  followed,  in  a  manner  similar  to  that  used  by  Karp.^  In  the 
path  matrix,  (Figure  8)  the  entry  in  row  F  colunm  G  is  fvf,  indicating 
that  the  program  can  arrive  at  G  by  one  of  two  alternative  paths. 
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IV.  STTECT  OF  LOOPS  ON  THE  DEBUGGING  SCHEME 


Noticeably  miaslng  from  our  example  flow  chart  (Figure  1)  is  any  loop 
that  causes  a  series  of  steps  to  be  repeated.  Such  circuits  occur  fre¬ 
quently  in  flow  charts^  but  their  discussion  has  purposely  been  delayed 
until  now.  Before  proceeding  to  their  consideration,  it  is  necessary  to 
clarify  the  aims  and  corresponding  requiienents  of  our  error  analysis. 

The  approach  to  be  taken  in  forming  a  complete  test  deck  is  to  consider 
conditions  such  that  every  path  will  be  traversed  at  least  once  and, 
furthermore,  that  it  will  be  entered  from  every  possible  entry  point.  If 
these  conditions  are  satisfied,  we  shall  know  that  the  work  performed 
along  a  path  is  correct  and  that  the  conditions  at  each  of  its  entry  points 
are  the  proper  ones  for  successful  completion  of  the  path. 

Under  this  premise,  it  would  be  unnecessary  to  go  through  each  loop 
more  than  once,  because  a  single  traverse  establishes  its  correctness. 

This  means  that  if  we  have  specified  conditions  to  cause  the  program  to 
exit  once  from  each  side  of  a  branch  point,  it  will  be  inconsequential  to 
us  leather  Che  program  subsequently  loops  back  to  a  previous  point  in  the 
program  or  goes  to  a  stop.  Consider,  for  example,  that  at  branch  point  D 
of  the  flow  chart  in  Figure  1,  path  d  loops  back  to  branch  point  A,  rather 
than  going  to  Stop  2.  If  we  simply  form  a  set  of  conditions  that  will 
cause  the  program  to  exit  from  D  on  path  d,  the  program  will  automatically 
execute  this  loop.  Furthermore,  all  of  the  paths  it  can  take  subsequent 
to  this  loop  will  be  identical  to  ones  we  have  already  accounted  for  with 
the  other  sets  of  input  conditions  that  we  have  pUnnad,  Ideally,  then, 
we  would  wish  to  have  a  set  of  conditions  that  would  cause  the  program  to 
exit  from  D  by  path  d'  the  first  time  it  arrives  at  D,  thus  omitting  the 
loop  from  its  sequence  of  steps,  and  a  second  set  of  conditions  that  would 
send  the  program  on  path  d  through  the  loop  exactly  once  before  it  exiti^ 
from  D  by  path  d ' .  Then  correct  answers  for  the  first  ret  and  incorrect 
answers  for  the  second  set  of  conditione  will  indicste  that  the  loop 
itself  is  at  fault,  whereas  incorrect  answers  for  both  sets  will  indicate 
that  the  error  probably  occurs  on  the  path  common  to  both  sets  before  the 
point  of  entry  to  the  loop  or  later  in  the  program. 

It  is  interesting  to  observe  the  effect  that  the  introduction  of  the 
loop  at  D  has  on  each  of  our  forma  representative  of  the  flow  chart.  It 
is  most  readily  shown  in  the  two  forms  of  the  matrix  (Figures  7  and  8), 
where  in  row  D  the  entries  at  the  intersection  with  columns  H  and  A  are 
simply  interchanged.  In  the  other  forms  of  representation  (because  of 
the  understanding  that  a  loop  may  be  considered  covered,  for  purposes  of 
debugging,  if  its  path  has  been  entered)  it  is  not  necessary  to  make  any 
change  whatever,  for  all  steps  preceding  the  loop  branch  were  traversed 
in  getting  to  it.  The  mere  fact  that  path  d  appears  as  one  of  the  paths 
in  the  representative  form  indicates  that  the  loop  will  be  entered.  Hence, 
in  the  tree,  path  list,  and  outline,  a  path  that  leads  Into  a  loop  will 
appear  the  sacK  as  one  that  leads  directly  to  a  stop,  so  far  as  the  de¬ 
bugging  requirements  are  concerned. 
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V.  CHOICE  0?  MATHEMATICAL  MODEL 


Careful  study  of  each  of  the  fonns  of  representation  of  problem  flow 
suggests  that  the  matrix  and  tree  can  be  most  helpful  In  error  analysis. 
Further;  the  tree  seems  to  be  the  simplest  form  for  the  programmer  to  use 
In  the  manual  generation  of  a  complete  teat  deck;  since  it  is  similar  to 
the  original  flow  chart  with  which  he  Is  accustomed  to  deeling.  By 
writing  the  program  In  terms  of  a  tree  with  each  loop;  as  well  as  each 
exit  to  a  program  stop,  designated  as  a  terminal  node;  the  number  of  test 
cases  required  for  exhaustive  check-out  can  then  be  determined  by  simply 
counting  the  terminal  nodes  of  the  tree.  The  programming  notation  for 
trees  presented  by  IverBon;I&/  and  the  ideas  used  by  Sauder^kL/  in  his 
general  test -deck  generator  for  COBOL,  encourage  the  possibility  of 
mechanisation  of  this  tPethod  of  error  analysis.  Matrix  representation  may 
be  found  preferable  for  machine  use  because  of  the  ease  with  which  a  matrix 
can  be  stored  In  the  machine  and  the  availability  of  the  lavs  of  matrix 
algebra  that  would  be  advantageous  in  the  analysis,  Since  it  is  a  mechanical 
matter  to  translate  ffom  tree  fomt  to  matrix  form,  the  progrananer  could 
still  do  his  part  using  the  tree,  and  then  ask  the  machine  to  translate 
the  tree  into  a  matrix  and  procaed  with  its  operation. 


VI.  TSST-DBCK  GStgRATIOM 


The  potential  number  of  test  cates  would  be  generated  by  considering 
the  effect  of  the  interaction  of  each  branch  point  with  every  other 
branch  point  in  the  program.  This  number  will  neve.-  become  a  reality, 
however,  because  it  will  always  be  possible  to  eliminate  the  consideration 
of  the  interaction  of  branch  points  between  which  there  Is  no  possible  path, 
auch  as  points  E  and  F  in  the  flow  chart  of  Figure  1.  The  logical  tree, 
drawn  either  directly  from  the  original  specification  of  the  problem  or 
from  the  flow  chart,  will  portray  the  complete  set  of  possible  Interactions 
among  the  branch  points,  each  different  combination  of  input  conditions 
being  represented  by  a  distinct  path  from  the  initial  node  to  a  terminal 
node  of  the  tree. 

The  mere  drawing  of  the  tree  automatically  simplifies  the  problem  in 
three  ways.  It  omits,  for  the  purposes  of  test-deck  generation,  all 
numerical  calculations,  all  paths  that  the  program  would  ordinarily  fol¬ 
low  subsequent  to  entering  a  loop,  and  all  unrealistic  Interactions 
between  branch-point  conditions.  In  addition,  the  tree  itself  may  often 
be  further  simplified  for  this  specific  purpose  by  giving  attention  to 
branches  for  which  the  input  would  not  differ  from  the  input  to  some  of 
the  other  branches,  i.e.,  where  the  same  test  criterion  is  used  two  or 
more  times  in  the  program.  For  a  set  of  branches  having  Identical  entry 


conditions,  all  but  the  earliest  of  these  branches  may  be  omitted  from  the 
tree,  because  test-deck  data  that  will  cause  entry  into  that  one  branch 
will  automatically  cause  entry  into  each  of  the  others.  For  example,  if 
Figure  1  depicts  the  flow  chart  for  a  business  problem,  the  decision  ele¬ 
ments  at  A  and  E  might  both  be  testing  whether  the  person  is  a  salaried 
or  an  hourly  employee,  the  paths  a  and  e  being  the  ones  that  will  be 
chosen  for  an  hourly  employee.  In  such  a  case,  input  conditions  that  will 
cause  exit  from  A  by  path  a  will  also,  without  exception,  cause  exit  from 
E  by  path  e,  so  that  the  decision  at  E  will  always  be  dependent  upon  the 
decision  at  point  A.  This  means  that  in  determining  test -deck  specifica¬ 
tions,  paths  e  and  e'  may  be  deleted  from  the  tree  in  Figure  3,  thus 
leaving  b'  and  c  as  terminal  nodes  and  reducing  the  number  of  test  cases 
by  two.  Such  repetition  of  test  conditions  throughout  a  program  is  not 
uncommon,  since  it  is  often  desired  to  inform  the  computer  at  different 
points  in  the  program  of  the  type  of  data  being  handled  in  order  that 
appropriate  computations  may  be  performed.  Before  reducing  the  tree  in 
this  manner,  however,  one  must  make  certain  that  the  input  conditions  are 
not  altered  in  some  way  during  the  course  of  the  program,  so  that  the  input 
to  the  branches  under  consideration  is  not,  in  fact,  identical,  After 
simplification  of  the  tree  has  been  achieved,  if  the  tree  is  still  too 
large  to  be  workable,  it  Is  helpful  to  postpone  the  consideration  of  the 
problem  as  a  whole,  in  preference  to  working  separately  at  first  with 
significant  portions  of  the  tree. 

When  the  working  portion  of  the  tree  has  finally  been  selected,  test-, 
deck  generation  is  begun.  Each  terminal  node  of  the  tree  stands  for  s 
diffarent- test  ease  Chat  should'  be  run  with  the  program  for  check-out. 

Test  data  are  selected  chat  will  cause  the  program  to  arrive  at  each  of 

these  end  points  exactly  once.  In  our  tree,  for  instance,  there  are  ten 

terminal  nodes,  indicating  that  there  will  be  ten  test  cases  for  our 
problem.  The  conditions  needed  for  any  particular  case  are  exactly  chose 
on  the  path  from  the  beginning  of  the  tr^e  to  the  particular  end  point. 

Test  Case  1  will  contain  data  that  complies  with  conditions  a,  b,  and  d; 

Test  Case  2  will  meet  conditions  a,  b,  and  d',  etc,  (Figure  3). 
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VII,  PEBUSQING 


When  a  list  of  all  possible  input  conditions  is  secured,  a  test  deck 
containing  an  input  card  for  each  condition  is  prepared  and  run  on  the 
machine  with  the  actual  program.  After  running  this  test  deck  on  the 
machine,  comparison  of  the  machine  answers  with  ones  that  have  previous  ly 
been  hand-calculated  will  Indicate  whether  there  were  any  errors  and  for 
which  input  data  they  occurred.  Kext,  a  quick  glance  at  the  tree  will 
often  give  a  very  good  idea  of  the  location  in  the  coding  to  begin  search¬ 
ing  for  the  error.  For  example,  if  the  answers  for  test  cases  3  and  4  were 
the  only  incorrect  ones,  the  error  is  very  likely  to  be  somewhere  in  path 
b',  whereas  if  1,  2,  3,  and  4  were  wrong,  the  error  is  probably  in  path  a. 
Likewise,  if  3  and  5  are  both  wrong,  but  all  others  right,  path  e  may  be 
at  fault  because  that  is  the  only  path  common  to  these  two  test  cases, 

The  aid  that  this  type  of  analysis  gives  to  pinpointing  an  error  is  par¬ 
ticularly  advantageous  to  the  prograisoar  who  seeks  to  find  the  error  by 
examining  a  monitor  sheet  showing  each  step  the  program  followed  for  a 
particular  case.  With  the  aid  of  the  tree  he  knows  where  in  the  monitor 
to  begin  searching  for  the  error,  and  if  there  are  two  errors  that  seem 
to  have  the  same  cause,  such  as  3  end  5,  he  can  compare  the  monitor  sheets 
for  those  two  cases  to  detenalne  whether  the  errors  truly  are  of  the  same 
type. 

If  it  is  necessary  to  work  with  only  a  portion  of  the  main  tree,  it 
will  be  found  that  after  this  portion  has  been  thoroughly  cheeked  out, 
its  test  deck  can  be  used  in  conjunction  with  ocher  portions  of  the  tree 
until  the  testing  of  the  entire  tree  has  been  completed.  The  use  of  this 
systenwtic  approach  simplifies  the  ^ork  coneiderably.  Since  each  test 
case  will  differ  from  the  next  only  slightly,  many  of  the  hand  calculations 
will  not  have  to  be  repeated  for  each  new  case.  It  would  be  most  valuable 
to  choose  the  test  date  close  to,  or  right  at,  the  limits  of  the  decision 
points  in  order  to  test  whether  the  branching  is  effected  exactly  as  it 
was  meant  to  be. 


VIII.  CHARACTERISTICS  TYPICAL  OF  SCIENTIFIC  PROBLEHS 


The  method  of  test  data  formulation  described  was  developed  originally 
in  connection  with  the  checking  of  a  buainess  application  of  computer 
programming,  The  method  and  this  type  of  problem  are  definitely  best 
suited  for  each  other.  In  a  business  problem,  there  is  usually  a  wide 
variety  of  input,  so  that  the  combination  of  varied  input  conditions  is 
one  of  the  major  problems  in  the  debugging  of  the  program.  In  addition, 
since  moat  of  the  branch  points  are  directly  or  indirectly  influenced  by 
the  input  data,  a  test  deck  with  input  data  varied  according  to  the  method 
described  in  this  paper  will  effectively  test  all  paths  of  the  program. 

In  a  scientific  problem,  on  the  other  hand,  the  input  data  generally  do 
not  vary  in  this  way,  and  the  majority  of  the  branch  points  are  influenced 
not  by  the  input  conditions,  but  by  the  inherent  properties  of  a  problem. 

A  loop  may  be  executed  a  certain  number  of  times  before  the  program  exits 
from  it,  and  the  number  of  times  may  not  be  influenced  in  any  way  by  the 
input'.  For  example,  in  computing 
10  ’ 

£  the  loop  that  accomplishes  this  summation  wili  be  repeated 

1-1 

exactly  ten  times  before  the  program  exits  from  the  loop,  regardless  of 
how  the  input  to  the  problem  la  varied.  If,  on  the  other  hand 
N  _ 

£  Xj_  is  desired,  and  N  Is^-variable,  then  the  value  of  K  will  be 
1»1 

included  in  the  input  data,  and  the  frequency  of  the  loop  will  he  controlled 
directly  by  the  input  conditions. 

For  some  scientific  cases,  the  r'nnning  of  one  test  case  will  completely 
check  out  the  problem.  In  such  a  case,  there  would  be  no  point  In  using 
the  tree  method  of  analysis.  For  a  problem  that  is  to  a  certain  extent 
Influenced  by  the  in^t,  there  are  two  possible  methods  of  dealing  with  the 
loops  that  are  independent  of  input  in  order  chat  this  method  of  analysis 
may  be  applied.  One  solution  is  to  omit  from  the  tree  the  decision  eliments 
for  iiuch  loops.  The  test  cases  would  then  be  United  strictly  to  the 
diverse  combinations  of  input  data,  The  assumption  here  would  be  chat 
since  the  loops  chat  have  been  omitted  from  consideration  will  automatically 
be  executed  In  the  process  of  the  program,  the  answers  to  the  test  data  will 
indicate  whether  these  loops  are  correct  and  are  traversed  the  proper  num¬ 
ber  of  times.  A  second  possibility  is  to  load  with  the  test  deck  a  sec  of 
pseudo-operations  chat  will,  for  the  sake  of  the  test  run,  afford  control 
over  the  branch  points  chat  are  not  normally  dependent  upon  the  input.  In 
10 

the  loop  for  T,  for  instance,  a  pseudo-operation  that  alters  the  branch 

1*1 

Ing  instruction  to  tept  whether  Xj^  has  been  summed  will  cause  the  program  to 


proceed  Itnoediately  upon  arrival  at  the  branch  point,  and  a  second  test 
case  containing  a  pseudo-operation  that  tests  whether  xo  has  been  sunraed 
will  allow  the  program  to  repeat  the  loop  exactly  once  before  proceeding. 
The  answers  for  the  two  test  cases  will  determine,  respectively,  whether 
the  sunmtion  procedure  itself  is  correct  and  whether  the  looping  is 
achieved  successfully.  The  advantage  of  running  two  such  cases,  rather 
than  checking  the  suBtaation  and  the  looplig  mechanism  simultaneously  by 
running  only  the  second^  is  that,  with  two  individual  cases,  the  exact 
trouble  spot  in  the  program  is  immediately  indicated  by  an  Incorrect  answer 
to  one  of  the  test  cases  and  a  correct  answer  to  the  other, 

With  the  addition  of  pseudo-operations,  as  described  above,  the  method 
of  error  analysis  using  a  logical  tree  is  no  longer  limited  to  input- 
dependent  decision  points.  If  the  complete  test  deck,  including  all  test 
data  and  pseudo -operations  indicated  by  the  tree,  IS  used  for  the  checkout 
of  the  problem,  there  will  be  exactly  as  many  test  cases  as  there  are 
terminal  nodes  of  the  tree,  The  incorrect  answers  that  appear  when  the 
complete  test  deck  is  run  will  enable  the  programmer  to  pinpoint  rapidly,  .  . 
using  the  tree  as  a  guide,  the  specific  points  of  the  program  that  are  in 
error.  If  the  prograioaer  prefers  to  spend  less  time  amassing  the  origins! 
test  deck,  an  alternative  method  would  be  to  use  only  the  various  combina¬ 
tions  of  input  data,  omitting  any  employment  of  pseudo -operations  for  the 
first  run.  Then  if  errors  appear  for  which  he  cannot  determi.ne  the  cause, 
he  can,  in  a  second  run,  use  some  pseudo-operations  for  the  area  of  the 
program  that  he  suspects  to  be  at  fault,  in  order  to  search  more  thoroughly 
for  the  error. 


IX.  CONCLUSION 

The  logical  tree  can  ba  invaluable  as  an  aid  to  sorting  out  the 
various  combinations  of  input  data  that  the  program  must  be  equipped  to 
handle,  testing  these  cases,  end  subsequently  assisting  the  programmer  in 
locating  the  cause  of  errora  that  occur.  Systematic  formilation  of  the 
test  deck  will  eliminate  the  inadvertent  duplication  of  teat  situations 
that  often  occurs  in  program  check-out,  end  will  significantly  Increase 
the  number  of  different  types  of  cases  tested.  It  is,  therefore,  an 
effective  aid  in  the  formulation  of  a  test  deck  that  will  contain,  for 
a  given  problem,  all  combinations  of  input  that  can  be  expected  to  occur. 
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